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A Method and Apparatus for Reformatting of Content 
For Display on Interactive Television 

5 

Copyright Notice 

A portion of the disclosure of this patent document contains material 
(code listings and message listings) to which the claim of copyright protection 
is made. The copyright owner has no objection to the facsimile reproduction 
10 by any person of the patent document or the patent disclosure, as it appears in 
the U.S. Patent and Trademark Office file or records, but reserves all other 
rights whatsoever. Copyright 2001 OpenTV, Inc. 

Background of the Invention 

Field of the Invention 

1 5 The present invention relates to the field of interactive television and 

specifically to the reformatting of content provided by a broadcast, Internet or 
cache for viewing on the interactive television display space. 

20 Summary of the Related Art 

Interactive television systems can be used to provide a wide variety of 
services to viewers. Interactive television systems are capable of delivering 
typical video program streams, interactive television applications, text and 
graphic images, web pages and other types of information. Interactive 
25 television systems are also capable of registering viewer actions or responses 
and can be used for such purposes as marketing, entertainment and education. 
Users or viewers may interact with the systems by ordering advertised 
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products or services, competing against contestants in a game show, 
requesting specialized information regarding particular programs, or 
navigating through pages of information. 

. Typically, a broadcast service provider or network operator generates 
5 an interactive television signal for transmission to a viewer's television. The 
interactive television signal may include an interactive portion consisting of 
application code or control information, as well as an audio/video portion 
consisting of a television program or other informational displays. The 
broadcast service provider combines the audio/video and interactive portions 
10 into a single signal for transmission to a receiver connected to the user's 
television. The signal is generally compressed prior to transmission and 
transmitted through typical broadcast channels, such as cable television 
(CATV) lines or direct satellite transmission systems. 



15 Typically, a set-top box connected to the television controls the 

interactive functionality of the television. The set-top box receives a broadcast 
signal transmitted by the broadcast service provider, separates the interactive 
portion from the audio-video portion and decompresses the respective portions 
of the signal. The set-top box uses the interactive information, for example, to 

20 execute an application while the audio/video information is transmitted to the 
television. The set-top box may combine the audio/video information with 
interactive graphics or audio generated by the interactive application prior to 
transmitting the information to the television. The interactive graphics and 
audio may present additional information to the viewer or may prompt the 

25 viewer for input The set-top box may provide viewer input or other 
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information to the broadcast service provider via a modem connection or 
cable. 

In accordance with their aggregate nature, interactive television 
systems provide content in various different communication protocols that 
5 preferably can be understood by the client or viewer who receives the 

information from the broadcast service provider/network operator. Typically 
the client is a set top box with a processor possessing limited processing 
power and communication bandwidth. Translation of the various protocols is 
beyond the limited processing capability available in the typical set top box 
10 processor. Moreover, there a multiple sources using a multitude of web 
authoring tools to create content. Thus there is a need for a robust server 
architecture, which receives and reconfigures the content from multiple 
sources so that it can be displayed by the client or set top box processor 
without an inordinate amount of processing or communication bandwidth. 

15 

Summary of the Invention 

The present invention addresses the needs of the interactive television 
environment discussed above. The present invention provides a method and 
apparatus comprising software and hardware for accepting a plurality of 

20 source content formats and reconfiguring the content for display on a client 
device. The content is preprocessed at the server so that reformatting and 
conversion at the client is not necessary. The preprocessed content also 
requires less bandwidth in a broadcast environment HTML code is 
transcoded into a concise compact code for display at the client device. 

25 MPEG imagery is reformatted for display on the client device. The viewer or 
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client can be a viewer at a (STB), a cellular telephone user, a digital assistant, 
a pocket PC or any other electronic reception device. 

Brief Description of the Drawings 
Other objects and advantages of the invention will become apparent 
5 upon reading the following detailed description and upon reference to the 
accompanying drawings in which: 

Figure 1 is an illustration of a head-end providing content to a client device; 

Figure 2 is an illustration of a head-end providing content to a client device; 

Figure 3 is an illustration of a head-end providing content to a client device; 
10 Figure 4 is an illustration of a head-end providing content to a client device; 

Figure 5 is a H20 HTML Transcoder architecture diagram; 

Figure 6 is H20 Browser processing and data flow diagram; 

Figure 7 illustrates an architecture for content translation, H20; and 

Figures 8-12 illustrates contend provision and display scenarios between the 
15 client/STB, H20 and application service providers; and 

Figure 13 illustrate conversion of a standard HTML image to a modulo 16 

MPEG image for display on a client device. 

While the invention is susceptible to various modifications and 
20 alternative forms, specific embodiments thereof are shown by way of example 
in the drawings and will herein be described in detail. It should be understood, 
however, that the drawings and detailed description thereto are not intended to 
limit the invention to the particular form disclosed, but on the contrary, the 
invention is to cover all modifications, equivalents and alternatives falling 
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within the spirit and scope of the present invention as defined by the appended 
claims. ■ 



Detailed Description of A Preferred Embodiment 

5 The following description describes several examples of preferred 

embodiments of the present invention. The present invention transcodes 
standard HTML into a concise, compact code for display of content on a client 
device. Thus, the transcoder offloads the task of interpreting HTML from the 
client to the head-end. The transcoder image can be significantly smaller than 

10 the original image. Thus, less processing power and communication 

bandwidth is required at the client The present invention extracts the size, 
color and form image information and sends these parameters to a layout 
engine. The present invention converts standard HTML/JPEG data into 
modified transcodes comprising MPEG images and location, size and color 

1 5 parameters. The transcoder initially parses the HTML input, but does not 
know how the page is laid out The present invention separates the location, 
size, color and form parameters from standard HTML input and uses these 
parameters to lay out the modified page to be displayed at the client device. 
Standard HTML is designed for page oriented, page specific 

20 presentation, wherein the HTML specifies precisely where each HTML 

element should be located on the page. The standard HTML input uses JPEG, 
PNG or other bit map oriented image formats to represent Web pages and 
content. The present invention uses MPEG and lay out parameters to 
represent images on a page for display at a client device. 



5 



WO 02/063878 PCT/US02/02664 

MPEG requires a module 16 position location, thus the present 
invention provides a buffer surrounding the MPEG representation of the image 
to insure that the MPEG image x and y locations are module 16 aligned. This 
modulo 16 buffer space, however, makes the image larger than original size. 
5 Thus, the buffer size is taken into account by the lay out engine so that the 
larger image size does not overlap text or other imagery adjacent the image. 
The MPEG representation enables the present invention to move MPEG 
images around on a MPEG background for display on the client device. Thus, 
the layout engine may not strictly follow the HTML layout to avoid overlap in 

10 the client display space. 

Standard bit map oriented representations merge all imagery and text 
representing a page, so that individual images within a page cannot be moved 
around in the page. The standard HTML/JPEG representation is also an order 
of magnitude larger than the MPEG representation. Thus, the transcodes 

1 5 generated by the present invention require less transmission bandwidth in the 
client display space. For example, a bit mapped image page for a 600 x 600 
pixel screen may occupy 500Kbytes of data. The same page can be 
represented as MPEG files in 30Kbytes and have true color. 

The present invention does not utilize standard Cascading Style Sheets 

20 (CSS) as normally utilized with HTML. The present invention converts the 
CSS into an HTML element node tree in a lay out engine. This is 
advantageous at the client, for example, to determine style locally at each 
node rather than referring to a CSS to determine the style or color of a node 
representing a modified HTML element The processor in the client has 

25 typically limited processing power and thus using the element mode tree, does 
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not have to go to the CSS to determine a style for a node for an HTML 
element The client does not have to determine what class a node belongs to 
determine a node's style. The present invention computes the root and puts 
the style, color, etc. with each HTML element node in the node tree. Thus, 
5 once at the client, the element node tree enables a client to manipulate down to 
each node and change the characteristics, for example, color, for a node, if 
desired. This is not easily done otherwise, for example, with Microsoft 
Internet Explorer, determining CSS class structure and attribute is compute 
intensive and inappropriate for the low processing bandwidth of the typical 

1 0 inexpensive client processor. 

The present invention also calculates the ratio variations required 
between standard HTML presentation and, for example, NTSC presentation 
on television or other client device. The ratio conversion is necessary so that 
HTML shapes are not distorted when displayed on a client device. Without 

15 ratio conversion, a HTML circle may appear as an oval on a client device. 

The present invention provides MIME handlers to read the JPEG and 
header for size, location and form. Another MIME handler performs the ratio 
conversion and converts JPEG to MPEG. The computed buffer zone for the 
MPEG imagery is calculated by a MIME handler and passed to the lay out 

20 engine. 

Thus, the present invention enables standard form content from a 
variety of content providers (Amazon.com, etc.) generated by a variety of Web 
authoring tools, to be displayed on a client device. The client display appears 
like a standard browser display, for example, Microsoft Internet Explorer (EE). 
25 The present invention enables display of this disparate content without 
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requiring the content provider to utilize special tools for formats to provide 
content to the client device, thereby simplifying provision of content to the 
client device. 

Turning now to Figure 1, the Service Platform SO comprises a group of 
5 applications roughly divided into three categories, Content Conversion 204, 
Transaction Control/Business Functions 206 and Transport Conversion 207. 
The Service Platform enables services 200 to interact with a client 212. The 
services 200 communicate through a communication link 202 to the Service 
Platform 50. The Service Platform 50 in turn communicates with a client 212. 

10 The client 212 may be a STB, a digital assistant, a cellular phone, or any other 
communication device capable of communicating with the Service Platform 
through communication link 210. The Content Conversion 204 and Transport 
Conversion 207 services provide the transport and communication function, 
and the Business Function services provide the Business Control functions. 

1 5 As shown in Figure 2, transaction Control/Business Functions 206 is' 

distributed between the Service Platform and the client 212. For example, a 
client can perform some business functions (e.g. implement advertising 
campaign rules and advertising/business filters to select advertisements 
viewed) and select contents, which are more suitable to the client 212 (e.g., 

20 select an advertisement or program which fits the user profile). The functions 
of Figure 2 are expanded in Figure 3. As shown in Figure 3, the Business 
Functions 206 comprise four major functional components: Service Manager 
238, Viewer Manager 240, Transaction Manager 242, and Advertisement (Ad) 
Manager 244. One example of a high-level operation flow for a preferred 

25 embodiment follows. 
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Referring now to Figure 3, a Service 200 negotiates with a network 
operator to offer a service to subscribers via the Head-End operators Service 
Platform. The network or Head End operator uses the Service Manger 238 to 
register the services and the negotiated business rules 222 (e.g. schedule, 
5 bandwidth requirements, service access to viewer information) associated with 
the service. The Service Manager 238 stores Service data 216 (e.g. URL 
address, content). Based on the business rules 222 and Service Data 216, 
Service Manager 238 communicates with the Broadcast Communication 234 
function to retrieve the content from the content provider. 

1 0 When the content is retrieved from the Service 200, it may be 

processed by the Content Conversion 204 and Content Filters 224 to convert 
the content into a form suitable for the client device 212. The Broadcast 234 
function converts the content into a form suitable for the broadcast 234 
network. The client 212 receives the converted content over broadcast link 

15 241 . Client 212 and Service 200 interact via Point-to-Point link 210 and 

Point-to-Point function 232, which are part of Transport Conversion 207. The 
service 200 may comprise shopping, audio/video, gaming, voting, 
advertisement, messaging, or any other service. 

Client 212 communicates through Point-to-Point 232 communication 

20 link to the Service Platform 50 and Service 200. Load Balancer 236 interacts 
with the Business Functions 206 to determine the optimal load distribution 
between the Broadcast 234 Communication link 241 and the Point-to-Point 
232 Communication link 210. The Platform Business Agents 226 use business 
rules 222 to control the interaction and exchange of information between the 

25 Service 200 and the client 212. For example, the network operator may 
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choose to prevent Service 200 access to user information. Service 200 
preferably pay a fee based on the Business Rules 222 and Service data 216 to 
access the user information. 

Viewer Manager 240 stores client/user information in User Data 220. 
5 Platform Business Agents 226 control the flow of viewer information to the 
Service 200. Transaction Manager 242 records transactional information 
exchanged between the service 200 and Client 212. Based on the Business 
Rules 222 and the User Data 220, Advertising Manager 244 determines which 
advertisements and which type of advertisements will be presented to the 

10 client via Broadcast 234 link 241 and Point-to-Point 232 link 210. 

Figure 4 illustrates another example of a preferred implementation of 
Service Platform 50. Services 200 provide shopping, chat, and other services 
either over the Internet or over another network or communication channel 
accessible to the network operator. Using the Service Platform, the network 

15 operator accesses those services. Business Functions 206, comprising Service 
Manager 238, interact with Carousel Manager 254 to retrieve content from a 
Service 200. The carousel comprises a repeating stream of 
audio/video/interactive data broadcast to clients from the Service Platform 50. 
Carousel manager 254, transaction manager 242 and Service Manager 238 

20 control the content insertion and deletion from the broadcast carousel. Service 
content is retrieved, converted into a client suitable format by H20 248. H20 
248 is a possible implementation of Content Conversion 204 and Content 
Filter 224. H20 converts HTML Content into Service Platform/Client 
readable content. The converted content is formatted into a data carousel and <> 

25 multiplexed by the Open Streamer 256 for broadcast to the client 212. Client 

IP 
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212 interacts with the services and if necessary communicates with the Service 
Platform and the Services 200. Point-to-Point communication goes through 
Service Gateway 246. Service gateway 246 performs transport conversion to 
convert the STB protocol into a form Platform Business Agents 226 and H20 
5 248 expect and understand. Load Balancer 236 interacts with Business 

Functions 206, Carousel Manager 254, and Service Gateway 246 to determine 
the optimal load between the Broadcast link 241 and the Point-to-Point 
Communication link 210. Business Functions 206, interact with the Platform 
Business Agents 226 to control access and information exchange between the 
1 0 Services 200 and client 212. 



H20 

In a preferred embodiment of the present invention, H20 is a 
client/server solution, which enables Internet content developers to build 

1 5 interactive TV applications and services for network operators running the 
Service Platform. The Service Platform enables viewer access to the larger 
pool of Internet talent and content made available to the vast growing 
worldwide market of interactive TV applications. The H20 server converts 
Internet content (HTML pages, ECMA Scripts, and HTML page formatting) 

20 into Service Platform assets. The H20 client renders the assets and interacts 
with the client. In a T-Commerce/E-Commerce case scenario, H20 enables E- 
Commerce shops to utilize existing Web tools to create shopping services and 
to interface with the preferred Service Platform (operator), using standard 
Web protocol. 
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H20 acts as a proxy to the Service Gateway and the broadcasting tools 
to convert Web content Web sites use their current HTTP servers and 
application servers to generate interactive TV content In a preferred 
embodiment, ECO converts HTML, scripts, e.g., JavaScript, and Internet 
5 graphics to client compatible code. Any other known or developed protocol 
can also be added to the functionality of H20. H20 enables the Service 
Platform to communicate with client's STBs that are not Ml browser capable 
and to create original user interfaces. H20 enables Service Platform 
connection to any commerce engine that uses only HTML. H20 is responsible 

10 for converting Web content such as HTML pages, JPG pictures, wave audio 
files, etc. into client device compatible resources that can be easily utilized 
with minimal processing capability and bandwidth at the client device. 

The server side of H20 is an HTTP proxy, H20S. For other purposes, 
it can be packaged as a DLL or batch tool. The client side of H20, an STB 

1 5 OCOD application, is H20C. H20C is built on top of other Service Platform 
client components, such as the Service Gateway library or the Carousel Load 
library. H20 enables URLs to be used to address documents and services. 
H20 enables tracking in the broadcast and online environments. H20S 
provides HTTP proxy functionality. Service Platform applications request a 

20 document through H20. H20 retrieves the document, parses it, compiles it, 
and returns the document to the requester. This H20 functionality enables use 
of the same engine for different uses, online and broadcast, facilitates 
scalability, and enables flexible use of H20. The parsing depends on the type 
of document, e.g., parsing can be HTML parsing, a GIF picture, or JPEG 
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images, etc. To make H20 expandable, H20 provides a function to "plug-in" 
and run new third party converters and filters. 

H20 enables scripting using different languages. Preferably, all 
Service Platform server components standardize around monitoring, especially 
5 the ability to remotely manage the different processes. SNMP is used to 
handle basic functions ("process OK", and traps on major problems). A 
command-line interpreter on socket for inspecting status is also provided. 
Setting parameters enables remote management and interface with the Web 
through Web scripts. In a preferred embodiment, standardized warnings and 

10 error logs are provided. 

H20 sends Email to the Service Platform Messaging Manager for error 
notification. Unsupported functions, tags, etc. cause the page to be rejected 
and an error message sent H20 enables controlled positioning of graphical 
elements (x, y, z). The output of Service Platform preferably comprises 

15 resources that do not duplicate assets in the broadcast H20 supports assets 
comprising, Imagery (JPEG, BMP, MPEG, GIF), Audio (set-top native) and 
Fonts; Animation (GIF), Binary (to be used by third party gadgets), CLUT 
(Fixed for an entire email) and dynamic control anywhere. 

H20 supports special tags comprising: A/V control, channel control; 

20 on screen display (OSD) control; and Triggers. H20 Supported W3C tags 
comprise: controlled positioning of graphical elements (x, y, z). JavaScript 
libraries are provided comprising Math, DOM partial, and Date. The client 
side of H20, H20C composes graphics assets in the client or STB. H20 
enables updated display of a user viewed page upon reception. H20C utilizes 

25 libraries (communication, carousel, et al.) are provided by other Service 
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Platform components. H20 enables refreshing an entire page in one shot, but 
also provides an option to choose between partial updates as assets are loaded, 
versus waiting for all assets to load. H20 enables dynamic linking/unlinking 
of third party classes. 
5 HTML/JS optionally does not share information from one page to 

another for the Web, as the server takes care of the context In broadcast 
mode, this arrangement is insufficient In broadcast mode, preferably, a global 
permanent object is provided, that is not cleared when starting a new page. 
The permanent object maintains context between pages. Other base objects 

1 0 provided by the Service Platform are also made permanent on transition (e.g., 
station control, OSD). Gadgets are defined through an interface definition 
language to enable creation of new gadgets, modification of gadgets and to 
enable adding methods without modifying the compiler. 

The H20 Carousel feature provides real-time updating of catalogs, 

15 maintaining consistency of catalogs during updates, and providing safe 
transactional models. H20 carousel enables updating a single page, or an 
entire set of pages in a single transaction. Carousel management provides 
management of a carousel index or directory. The index contains information 
• for accessing and fetching data from the carousel. That is, for a given page, 

20 Carousel Management contains a list of all modules necessary, so that H20C 
requests all necessary modules at once to expedite the process. 

Carousel provides data compression, Meta data on pages (e.g., page 
relative priority, how often the page is sent) and page tracking (elementary 
stream). The Carousel client is a STB OCOD library, handling the loading of 

25 resources. Carousel client manages dynamics of resources, i.e., new 
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resources, deleted resources, and changed resources. Dynamic resource 
management enables the client (H20C) of this library to present dynamic 
content The Carousel client manages memory allocation, pre-fetching and 
caching of resources, and decompression of modules. The Carousel client 
5 manages sub-index/directories and manages a 'set 1 of resources instead of a 
'tree' of resources, which facilitates sharing of assets. 

H20 monitors TV triggers, performance, and bandwidth, e.g. shared resources 
in shared modules. H20 optimizes bandwidth utilization. H20 provides 
multi-tracks, multi-priorities, and management of bid volume of data. H20 

10 provides run-time pre-fetching and caching at the module level. H20 supports 
compressed modules. Surfer built-in GUI is minimal or preferably zero. H20 
' supports arrow and direct key navigation (e.g. digit or color), Handling 
international (Chinese); Meta data on pages, (e.g., page relative priority, how 
often it is sent) and page tracking (elementary stream). Global GUI is shared, 

1 5 that is, a direct key linking is provided so that every other page can share any 
information page. 

H20 manages pages and sub-pages to handle cases where pages are 
too large to fit on one screen without the need for scrolling. H20 enables use 
of HTML to present content, online, point-to-point, and broadcast H20 

20 enables composition of a page with a mixture of broadcast and online 

components. For example, a page can come from an online server, while its 
background is broadcast H20 enables merger of content in the client/STB. 
For example, a bank application can send a viewer's last 20 VISA® 
transactions from its server while the HTML page is broadcast Preferably a 
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Java Script function request (HTTP like), using the result and some DOM 
functions patches a table with the result. 

Now turning to figures 5 and 6, a discussion of the main H20 
components follows. H20 transcoder 2001 converts HTML inputs 2062 into 
5 H20 transcodes that can be efficiently and interactively viewed by an H20 
Browser in OpenTV STB. The H20 Browser views H20 transcodes in 
limited Dynamic HTML4 feshion. The host, such as HTTP proxy 2003, 
invokes H20 Transcoder 2001. H20 Transcoder 2001 interfaces with MIME 
handler 2036 and MIME info 2028 for content information in non-HTML 

1 0 MIME type. The resource compiler is multi-thread safe. H20 Transcoder 
invokes H20 JavaScript Handler 2000 to take scripts as input, fetch external 
scripts or class URI, invoke JS20 compiler and pre-link JavaScript. The JS 
handler processes JS custom classes 2070 and JS shared scripts 2066. 
The H20 Browser 2100 interfaces with JS20 JavaScript runtime 2129 for 

1 5 JavaScript support. H20 Browser interfaces with URI Retriever 2162 to 
retrieve content by URI. H20 Browser 2100 typically uses URI Retriever 
2162 to make HTTP URI request and get response, and to handles cookies. 
H20 Browser uses image library, and event library. H20 transcoder is 
invoked as a library as part of the task (process or thread) in a high 

20 performance HTTP proxy. H20 transcoder contains a HTML parser/layout 
engine source as its major component In the context of that hosting 
environment (proxy server) received HTML URI requests and fetches HTML 
text source, the H20 transcoder provides an interface for hosting the 
environment to control. The H20 Interface comprises: H20 Jxanscoder; 

25 method: H20 S_generate Jranscode. Input: memory block of MIME type 
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text/html, target view. Return: memory block of transcode type. Behavior: 
Parse, layout input HTML content and generate transcode type, which is ready 
to be used by H20 browser running on the client device, preferably a STB. 
H20 transcoder executes synchronously and is multi-thread-safe. 
5 When processing HTML input, the H20 transcoder discovers all URIs 

that are statically referenced and in the HTML input The H20 transcoder 
uses URI handler 2034 to fetch the list of URIs and references needed and 
returns information that the H20 transcoder uses. The H20 transcoder uses 
the URI content information, that is, the width and height of the information to 

10 be used in the layout process. 

The MIME interface handler is exposed to the H20 transcoder as 
follows: Interface: MIMEJhandler; method: MIMEHandler_handle_uri; 
Input: URI string with accepted MIME type, view option, processing options. 
MIME type: pixmap, bmp, gif, jpeg, font, color table, JavaScript text, MPEG 

15 audio, SP code module and data resource module. Return: MIME content 
information, not necessarily the full content; Image usually contains 
information of type, height, and width for the layout engine. 

Behavior: Rim as a task, fetch MIME content through host HTTP 
proxy, process MIME content as requested. Preferably an interface is 

20 provided with proxy host to handle the caching mechanism for better 
efficiency. The H20 transcoder executes synchronously. 
A subset of HTML4 features is supported in H20 transcoder. HTML4 content 
is rendered by the transcoder to a particular target view, in the rendering 
process almost all rules of HTML elements and attributes are applied and style 

25 rules including default styles are computed. The dynamic features of HTML4 
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(mainly style and DOM interface) are restricted through limited DOM 
interfaces in the present invention. 

The H20 Browser supports a selected subset of the HTML4.01 W3 
standard. The H20 browser supports a subset of text paragraphs, lines, 
5 phrases; subset of list; subset of table and subset of links. The H20 browser 
also supports a subset of Object and Image element The H20 browser also 
supports a filter authoring-side object (e.g., IE custom object) and supports 
runtime custom objects for the STB; both authoring-side (PC) and STB 
custom object class development; subset of image map (client-side, server- 
10 side); subset of form and form controls; various Script elements; and a subset 
of JavaScript 

The present invention provides functionality that defines color map 
extension elements. The present invention provides functionality that defines 
client tags extension, supports a subset of CSS and limits CSS, mainly for 

1 5 viewing options for a client display, preferably a TV screen. The present 

invention supports a subset of W3 standard CSS in the H20 transcoder. Most 
CSS features are supported in H20 Transcoder. The dynamic features of CSS 
are restricted in the H20 Browser. The H20 Transcoder applies the CSS to 
the document layout and presentation. In H20 transcoder, the HTML nodes 

20 contain once-computed CSS information in element nodes. Very limited, if 
any, additional CSS is preserved. This is fundamentally different from the 
dynamic cascading rules of CSS. Dynamic style is limited as a style change 
preferably is applied to only one node. This means that in order to query 
through DOM a property of the style of a particular element and expect it to 

25 return a valid value, the style property (e.g., style= color: red) is explicitly 
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specified for the element inside an inline style, or explicitly created in a 
JavaScript code in order for a PC emulation to behave similarly. 

The present invention supports top, bottom, left, right absolute 
positioning styles, but preferably does not support z ordering, although, in an 

5 alternative embodiment, z-ordering is supported. The H20 transcoder defines 
target STB style for the layout engine comprising body margin, view style 
(OSD region and dimension, width limitation), mpeg background or not, and 
transparency rules. The Mpeg slice inlage in the layout flow is padded with 
extra layout bounding box size to accommodate the 16x1 6 block alignment 

1 0 Mpeg requirement, which does not conform to typical browser layout fashion. 
The present invention defines style rules/attributes by configuration and 
explicit specification to specify conversion of content to an MPEG 
background and slice from other MIME type content The present invention 
resolves multi-line anchor information in server side using only a first line 

1 5 anchor or image as an anchor. The H20 transcoder generates style information . 
for focus highlight information. Limited style information is stored in 
transcoder. 

The present invention supports a subset of HTML DOM in the H20 
Browser. The present invention also provides for HTML parsing exception 
20 handling. 

The present invention enforces a strictly defined HTML subset either 
by the parsing engine or in a separate validation module. The HTML parser 
engine handles invalid tag order, tag closing, tag nesting, and attributes with 
acceptable appearances. The HTML parser engine's exception handling 
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(resulting in element tree difference, layout differences, etc.) is different from 
Microsoft IE and other standard browsers. 

The H20 Transcoder controls the processing behavior of the HTML 
engine. The H20 Transcoder can fully access the generated data model of the 

5 parsing/layout processing. The HTML engine is reentrant safe and can be run 

i 

as a library. The present invention invokes the HTML parser engine to parse 
HTML source and generate the engine's internal document representation 
(OOM). The parser engine is modified to conform to the H20 HTML/CSS 
functionality specification. The present invention controls the internal parsing 
10 to process elements based on a HTML4 subset definition. The present 

invention enables, disables, extends, and modifies HTML element parsing. 
The present invention adds required information into internal data model or 
engine (e.g., width and height of image). The present invention also handles 
script elements. 

1 5 The present invention invokes a H20 JavaScript handler task to 

process Script element data. The JavaScript handler task fetches the script 
URI, and invokes the JS20 JavaScript compiler. JS20 returns a data package 
containing the compiled JavaScript code module. JS20 caches JavaScript 
scripts and matches repeatedly similar JavaScript scripts. 

20 The preferred layout engine lays out an internal data model for target views by 
configuration and explicit specification in the HTML document. The layout 
engine is modified to conform to the specified H20 HTML/CSS functionality 
specification. The present invention resolves multi-line anchor issue on the 
server side and uses only a first line anchor or image as anchor. The 

25 transcoder generates computed layout/style information and attaches it to the 
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internal data model, of generates a data model. The present invention layout 
applies suggested width, but does not force limited width on layout 
The present invention generates H20 Transcode using client 
compatible resources from parser engine internal data models with layout and 
5 limited style information, and JavaScript codes. A general specification of 
H20 Transcode format comprises Transcode sections of client compatible 
resources that represent the document and JavaScript codes for the client or 
STB. The Transcode format contains information about the document in tree 
structure. The format is efficient, compact, and expandable. The format 

10 provides a version number to ensure conformance in the client or STB. The 
format considers compression and relocation. The efficient format enables 
efficient loading by the client or STB browser. The format contains enough 
information for the H20 browser to efficiently resolve linkage. The Version 
of package format is preferably synchronized with all related products, such as 

15 H20 browser. 

O-code HTML browser 

The H20 Browser 2100 views H20 Transcodes in limited Dynamic 
HTML4 fashion. H20 Browser interfaces with JS20 runtime module 2129 
and contains internal components: OOM (Client compatible Object Model), 

20 Controller, and level 2 HTML4 Data Model. 

The HTML browser supports a subset of the HTML DOM standard. The 
HTML browser supports HTML DOM node traversal, creation, destruction, 
and insertion and supports element lookup by unique ID. ' The HTML browser 
resolves object initialization and references during insertion. The HTML 

25 browser supports node cloning. The HTML browser preferably does not 
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provide dynamic parsing of HTML script to transform into part of DOM. (e.g., 
document writeO). The HTML browser defines the order document view is 
updated to reflect changes to HTML DOM object The preferred HTML 
browser does not support dynamic re-layout of layout flow in STB to reflect 
5 modification of HTML through DOM. Changes made to layout flow text do 
not result in re-layout, which wraps text. Layout flow text is preferably not 
wrapped. Though no changes are made to layout flow text, changes to text 
property in node objects such as Text area (text can wrap) or Input can be 
updated in view. These achieve the effect of dynamically updating UI objects. 

10 The HTML browser provides subset of HTML DOM interfaces and objects 
for JavaScript to manipulate HTML DOM objects. The HTML browser 
provides the same support for a C program. The HTML browser handles 
HTML DOM events and provides a mechanism to map an authoring side font 
to client compatible font The H20 DOM interface manipulates client 

1 5 compatible font information. 

OOM (Client Compatible Object Model), 

The OOM is provided for a developer to develop custom classes. The 
OOM provides a mechanism to develop custom object class, load/register/un- 
register/unload class, and access class interface code to manipulate custom 

20 objects. The design and implementation of OOM uses JavaScript object 
model to avoid duplication of effort. OOM provides interface for both 
JavaScript's and C programs to manipulate objects. HTML, OOM, and C 
programs function without JavaScript. The present invention provides 
lightweight client-side class registry with selected naming scheme. The 

25 registry enables associating a class module with a name and lookup. OOM 
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loads a class module and registers the module. There is only one interface for 
a class. Object prototype property is not dynamically bound. The interface 
contains methods only. Each method is associated with a string name unique 
in this interface scope. OOM provides a mechanism to create an object 
instance from a class and a Provide class referencing mechanism; OOM 
handles class unloading. OOM defines control flow, data flow between H20 
HTML and JavaScript engine. OOM instantiates browser objects for the 
JavaScript engine. OOM exposes access of DOM objects to the JavaScript 
engine (through setting properties of global variable property of window 
object with implementation tuning). An object of a class is an object whose 
method properties are methods of the class interface. OOM defines interfaces, 
guideline, subset of SDK API for custom object to be integrated into view 
structure, view update, event handling, focus, and access to shared system 
resource. 

A controller controls Browser behavior and components, and handles 
events. The controller provides fast startup time response, achieved by 
various techniques such as visual displaying while starting up. The preferred 
controller of die present invention uses system resources: event, download, 
mpeg, osd, A/V, etc. and controls data model and viewing. The Controller 
handles raw OpenTV events 2146. The controller handles primitive object 
events, which in turn can generate a DOM event 2140, 2134. The present 
invention supports DOM event 2138 bubbling and handling and uses focused 
node as a starting node. DOM events can be handled by JavaScript event 
handler 2138 codes. 
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A focus navigation handler 2148 is provided by the present invention 
that handles focus event using document tree, event properties, and style 
(rectangle), and handles ambiguity. Record focus history by object identifier 
and x, y position, and finds the best match. The controller preferably defines 
5 control flow of event dispatching, event handling, data update, and view 
update. 

The preferred controller pre-defines element classes and thus is more 
efficient than custom classes. The controller using URI retriever, obtains URI 
content, makes HTTP: get and post (form post) requests, obtains responses 

10 and provides document open and back/forward functions. In a preferred 
embodiment, Data Models are mainly runtime HTML DOM, and other 
categories of runtime data models. HTML DOM also contains viewing 
information that reflects their intended effects. 

The Data Model handles one single runtime HTML document, which 

1 5 can be instantiated from document sources, and the previous one is destructed 
The data model supports preferably 

load/instantiate/reference/manipulate/destruct data models. The present 
invention provides persistent data save/read to serialize data and share data 
between documents. In a preferred embodiment, viewing actually reflects the 
20 UI looks of data models, and mainly reflects the actual style of HTML DOM. 
The viewing functionality is tightly embedded into the data model, mainly 
using internally defined shapes. The HTML page specifies exactly what 
viewing option it uses. The present invention defines specific viewing 
limitations such as screen dimension and ratio, color, and mpeg 
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characteristics. There is one preferably view for one runtime HTML 
document 

Viewing preferably supports different color styles, and treats the color 
table as a shared resource. In a preferred embodiment, a limited set of MIME 
5 type handlers are provided depending on configuration to save memory. 
Popular HTML browsers for PC environments typically expose browser 
specific host objects for JavaScript runtime to manipulate. The present 
invention supports a subset of Microsoft IE5 host objects for JavaScript 
runtime. 

10 The transcoder invokes the parser engine as follows: 

HTMLengine _parsejhtml (C_MIME_HTML *p_injitml_text, 
CJBLEMENTTREE **pp_out_elementJree). C_ELEMENT_TREE 
encapsulates the internal element tree generated by parser engine. In parser 
engine, find right approach to disables APPLET, BDO, FBELDSET, FRAME, 

15 FRAMESET, IFRAME, ISENDEX, LABEL, LEGEND, Q, S, STRIKE, U. 

The parser engine, capability is provided to parse extension elements 
and generate extension element node. Internal client compatible extension 
elements comprise: link attributes for carousel, display Meta (NTSC/PAL, 
MPEG background/slice, OSD region), color table style (color bits, color 

20 name/index mapping), font (read font info from OpenTV font file). 

The transcoder discovers and requests URI references for layout as 
follows: 

int HHTM_resolve_referenced_uri (C_ELEMENT_TREE 
*p_in_element_tree); 



25 



WO 02/063878 PCT/US02/02664 

The preferred transcoder calls int MIMEJmageJiandler (MtMEJTYPE 
injype, char *p_in_uri, URIJNFO **ppj)utjiri_info). Preferably, the 
transcoder traverses the element tree and when necessary invokes MIME 
handlers with URI and handling options (e.g., whether to and how to convert 
5 an image to a mpeg slice with dimension and NTSC/PAL ratio options), and 
fills in information in element nodes such as width and height of image, font, 
and color table. For images that will become a MPEG slice and its position 
not determined, the present invention expands the image element attribute size 
to achieve module 16 pixels on both width and height 

10 . The present invention invokes H20 JavaScript type handler. TheJS 

handler traverses the element tree to collect JavaScript global code, functions, 
link reference of external script, object class link reference, and host codes 
(from event handler attribute). Invokes H20 JavaScript handler and pass this 
collection. The JavaScript handler task fetches script and class MIME 

1 5 content, and invokes JS20 JavaScript compiler which returns a client 
compatible code module containing the compiled JavaScript codes. The 
script collection carries back handles to codes. The function is invoked as 
follows: 

int HHTM_collect_script (CJBLEMENTJTREE *p Jn_element_tree, 
20 C^SCRIPT^COLLECnON **pp_outjscript_collection); 
HJS: H20 Transcoder JavaScript type handler, 
int mS_handlerJiandle_script(C_SCRIPT.COLLECTION 
*p_in_script_collection, 

C JS_CODEJdODULE **pp_outJs_code_module); 
25 The layout engine in invoked as follows: 
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Int HTMLengine Jayoutjitml (C_ELEMENT_TREE *pjn_element_tree, 
CJDW_STYLE_RULES *pjmjiefault_style); 

The layout engine uses information in the element tree and its own 
style configuration mechanism for layout Default styles are transferred into 
5 the layout engine's style configuration and layout target view. Style 

configuration can take the form of callback, which requests style information. 
The layout engine stores layout result in element tree. The layout engine may 
traverse the element tree and generates another data model such as style box 
model. The layout engine intercepts the computed CSS information for a 
10 specific element node, finds the corresponding element node, and stores the 
style result in specific element node as inline style with limited cascading style 
information. 

The layout result includes necessary style information (position, color, 
font, background). Position styles are translated into absolute position styles. 

15 The layout engine traverses all anchor information. If necessary, the layout 
engine modifies the anchor information to use only the first box or image as 
anchor. The layout engine traverses and marks leaf nodes that can interact 
(can handle click, keypress, blur, focus events) as focusable. 

The transcoder traverses the element node tree and generates H20 

20 Transcode using the internal client compatible resource generator. The 
present invention configures components, display, startup, etc. Static and 
dynamic configuration is statically determined at compile time. The present 
invention initializes the H20 Browser. It first initializes the controller and the 
controller then initializes OOM, DOM, JS20 Runtime, register event loop and 
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event handlers, initializes URI Retriever, initializes image library, reads view 

option, and initializes host objects. 

The present invention provides DOM built in classes. Root classes are 

implemented as collections of functions, and all DOM node object instances 
5 have common data members that are meant for root class to operate. The 

function pointers are obtained by calling the vtable pre-defined function that 

switches on a method atom query and returns the function pointers. Event 

handler is a list of JS event handler function pointers referenced by event type. 

Specific control element classes are implemented as a collection of functions 
10 and all DOM node object instances have common data members that are 

meant for root class to operate. 

The following is an overview of the H20 Proxy environment using a 

logical view of H20 architecture and sample transactions. Requests for URI 

may come either from different H20 components - for example: STB/SGW 
1 5 and Carousel. The following context overview focuses on the STB/SGW 

issuing the requests - but the general flow of information stays the same. 
A viewer chooses to interact with its TV Web page, thus issuing a 

request from the STB to the H20 system and waiting for a reply. STB 

requests are sent to the SGW, using lightweight HTTP requests (LHTTP) 
20 encapsulated in DATP messages as transport protocol. The requested object is 

returned through the same channel and protocol. The SGW converts the 

LHTTP protocol to standard HTTP over TCP/IP and issues the request to a 

Web Cache. 

The Compiled Object Cache (COC) uses its internal disk space to service any 
25 request it can serve (following a heuristic taking into account the time-to live 
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of objects). Its role is to service all static objects (standard URLs without 
queries, no posted form) without querying the H20 proxy, thus reducing it's 
processing load. In this architecture, the COC will only store compiled objects 
(H20 modules). The COC machine is I/O driven. 
5 Now turning to Figure 7, H20 proxy 248 provides a scalable 

environment for the different H20 compilers (or filters) to run. It is 
processing HTTP request and responses "on the fly" and thus the H20 Proxy 
machine is process driven. The H20 HTML Compiler 1420 is in charge of 
HTML to SP resource compilation. To compute the TV Layout to render 

10 1422, this component issues HTTP requests by itself based on the size of the 
embedded images. This compiler rearranges the Web based image to fit onto 
the client display device, e.g. a TV. 

The MPEG Compiler 1426 directs the conversion from regular web 
image format to SP H20 MPEG resources. Source format includes JPEG and 

15 GIF and may include PNG. Passing arguments through the URL may drive 
the conversion process. The PDCMAP Compiler is in charge of the conversion 
from regular web images to SP H20 resources. Source format comprises GIF 
and may include PNG. 

The Request Patcher 1424 completes or modifies the request or 

20 responses to incorporate data coming from another system (e.g. credit card 
number. . ,). It communicates with an external process or database to fetch 
customer information. The SP component provides a central repository of 
user information. The Request Patcher communicates with this component to 
get the data necessary to patch the requests or responses. 
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The Not Compiled Object Cache 1430 will use its internal disk space 
to service any request he can serve (following an heuristic taking into account 
the time-to live of objects). The objects cached comprise static HTML, GIF 
images, JPEG images and all standard web format files. Its role is to service 
5 all static objects (standard URLs without queries, no posted form) without 
querying the Internet, thus reducing latency time to get an object and giving a 
kind of fault-tolerance to the system. The Customer web site holds the web 
site to publish through the ECO system. 

Figure 8 illustrates a request for a static page, already cached The STB 

10 user issues a request to load an HTML page 1520. This request is sent to the 
SGW 248 using LHTTP over DATP. The SGW converts the request to HTTP 
over TCP/IP and forwards it 1522 to the Compiled Object Cache 1410. The 
Compiled Object Cache 1410 has the requested (compiled to a module) 
HTML page stored in its internal hard disk space. If the object's time to live 

1 5 has not expired and the compiled object cache service the request with the 
compiled HTML page. It transmits the HTTP response 1424 to the SGW, 
using HTTP over TCP/IP. The SGW translates the protocol from HTTP over 
TCP/IP to LHTTP over DATP. The STB loads the requested page 1526 
(compiled) in its memory and gives it to the H20 browser engine for 

20 interpretation. The H20 browser engine requests 1528 the SGW to get the 
images necessary to render the screen on TV, with conversion options (mpeg 
or pixmap, width, height. . .) on the url. The SGW transmits the HTTP request 
1530 to the Compiled Object Cache. The Compiled Object Cache has the 
requested (compiled to a module) image stored in its internal hard disk space. 

25 The objects time to live has not expired and the compiled object cache 
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services 1532 and 1534 the request with the compiled image. In this scenario, 
the H20 Proxy was spared of the request and thus can process other requests. 

As shown in Figure 9, the STB 212 user issues a request 1610 to load 
an HTML page (home.asp), the host and user info of the request header hold 
5 STB Model+ STB Serial number and Access Card Id of the user. This request 
1610 is sent to the SGW using LHTTP over DATP. The SGW converts the 
request to HTTP over TCP/IP and forwards it 1612 to the Compiled Object 
Cache. The requested object is not available in the disk space of the Web 
cache. The Web Cache then forwards the request 1614 to the H20 Proxy; The 

1 0 H20 Proxy 1616 asks the SP 1620 to return the name of the user (for the 

Amazon.com service). The H20 Proxy patches the request with the name of 
the user, and issues this request 1622 to the **Not Compiled Object Cache". 
The "Not Compiled Object Cache" does not hold the requested HTML page in 
its disk space and thus issues the request 1624 to the targeted web server, here 

1 5 Amazon.com. The targeted web server computes the HTML page, given the 
user information and returns it to the "Not Compiled Object Cache 1626". 
The 'Not Compiled Object Cache" returns the HTML page 1628 to the H20 
Proxy. 

The H20 Proxy sends the HTTP request 1630 to the "Not Compiled 
20 Object Cache" to get the images 1632, 1634, and 1636 necessary for layout 
computations (gif, jpeg. . .). The H20 Proxy compiles the HTML page; 
computes the layout, patches the embedded images urls and sends back to the 
"Compiled Object Cache" the resulting OpenTV resource 1646 (with an SP 
resource mime-type). The Compiled Object Cache stores the object in its 
25 internal disk space and sends back the compiled HTML 1648 page to the 
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SGW. The SGW converts the response to LHTTP over DAT? and sends it 
back to the 1650 STB. The STB loads the requested object in its memory and 
gives it to the H20 browser engine for interpretation. 

The H20 browser engine issues requests 1652 to the SGW to get the 
5 images necessary for the rendering (through the patched urls: here the logo.gif 
url include a directive for pixmap resource format): pix/logo.gif. The SGW 
converts the request 1652 to HTTP over TCP/DP and forwards it to the 
Compiled Object Cache. The "Compiled Object Cache" already has the 
requested gif image, in the correct resource format - because a user already 

10 requested this image at a previous time - and the image is directly returned 
1654 to the SGW. The SGW converts the response to LHTTP over DATP and 
sends it 1656 back to the STB. The H20 browser engine issues requests 1658 
to the SGW to get the images necessary for the rendering: mpg/banner.jpg. 
The "Compiled Object Cache" does not hold the requested image in its disk 

15 space and therefore issues the request 1660 to the H20 Proxy. The H20 

Proxy sends the HTTP request 1662 to the "Not Compiled Object Cache" to 
get the /banner.jpg image. 

The "Not Compiled Object Cache" holds the image in its cache and 
returns it 1664 immediately to the H20 Proxy. The H20 Proxy converts the 

20 image, using the parameters given in the url (mpg format, width, height. . .) 
and returns the result to the Compiled Object Cache 1668. The Compiled 
Object Cache stores the object in its internal disk space and sends back 1668 
the converted mpeg image to the SGW. The SGW converts the response to 
LHTTP over DATP and sends it back 1670 to the STB. The STB renders the 

25 HTML page to screen. . 
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The H20 Proxy component provides to other H20 components or 
compilers, a robust scalable architecture, an interface for "compiler" 
configuration and the ability to log errors. The ability to alert an administrator 
on defined events; and the ability to debug-trace the "compilers". From the 
5 provided H20 Proxy environment and APIs the compilers "patch" HTTP 
requests and responses on the fly, eventually accessing an external database, 
file or process to do so. The compilers patch HTTP requests by removing 
specific HTTP Header (STB identifier, Access Card Identifier...); by adding 
specific HTTP Header (Username, Credit Card Number. . .); by adding HTML 

1 0 Form fields to incoming post request (Visa Card number. . .); and by 
performing string substitution ($UH>$ -> User Identifier) the compilers 
convert web objects formats and mime types "on the fly" in HTTP responses 
and issue HTTP requests by themselves and get a response object in return. 
As shown in Figure 10, in a preferred embodiment, the H20 Proxy is 

1 5 implemented by developing an extension of enclosing software (W eb Proxy, 
Firewall, web server or other. . .). This host software provides H20 threading 
and scheduling of the H20 tasks as well as some of the needed functionalities 
to implement H20 "compilers" and patching components. 

Using the API provided by the Proxy Host Software, a set of API (the 

20 H20 Proxy API) is provided to implement the functionalities needed by the 
H20 compilers missing from the H20 Proxy Host Software Services, and 
provide a higher abstraction level for the services available from the H20 
Proxy Host Software. The request patcher 1424 component reads incoming 
HTTP requests for HTML pages and completes them with information from 

25 another process or file or database. The HTML2RES Compiler 1420 compiles 
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returned HTML pages into SP resources files and change the mime type of the 
HTTP response header to suit new format: Mime-Type: text/otvres. 

The GIF2PIX Compiler 1422 converts a returned GIF image into an 
SP resource file and changes the mime type of the HTTP response header to 
5 suit new format: Mime-Type: image/otvpix. The 2MPEG Compiler 1426 
converts a returned GIF or JPEG image into an SP resource file and change 
the mime-type of the HTTP response header to suit new format: Mime-Type: 
image/otvmpg. 

Turning now to Figure 11, a dynamic request for an HTML page 

1 0 sequence diagram is illustrated The Object Caches are not displayed in the 
Sequence diagram, being "passive" components in this interaction. The User 
STB 212 issues a request 1810 for a page (home.asp) through HTTP request 
The Request Patcher 1424 accesses an external process/file/database/url 1812, 
1814 to get user name, patches the request and sends 1816 it to the 

1 5 HTML2RES Compiler. The HTML2RES Compiler sends the request 1818 to 
the target web site (amazon.com). The Web site computes the request and 
sends back 1820 the resulting HTML page to the HTML2RES Compiler. The 
HTML2RES Compiler parses the file to get the image links URL and issue the 
requests 1822 to the web site to get 1824 the image'files (logo.gif, banner.jpg). 

20 The HTML2RES Compiler computes the TV layout for the page, compiles it 
into SP resource file, and sends it 1830 back to the STB. The STB renders the 
HTML page on TV. 

Turning now to Figure 12, a request for an image file, sequence 
diagram is illustrated. An HTML page being loaded in the User STB needs 

25 an image to render its screen. It issues an HTTP request 1910 for the image 
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(URL embedded conversion options) to the 2MPG Compiler. The 2MPG 
Compiler requests the image 1912 from the target site (amazon.com). The 
target site returns the banner.jpg image file 1914 to the 2MPG Compiler. The 
2MPG Compiler converts the banner.jpg file, using the options given on the 
5 URL and returns the result 1916, with an image/otvmpg mime-type, to the 
STB. The STB renders the image on screen. 

As shown in Figure 13, Standard HTML 1302 is designed for page 
oriented, page specific presentation, wherein the HTML specifies precisely 
where each HTML element should be located on the page. The standard 
10 HTML input uses JPEG, PNG or other bit map oriented image formats to 
represent Web pages and content The present invention uses MPEG and lay 
out parameters to represent images 1304 on a page for display at a client 
device. 

MPEG requires a module 16 position location, thus the present 
1 5 invention provides a buffer surrounding the MPEG representation of the image 
to insure that the MPEG image 1304 x and y locations 1310 and 1312 are 
adjusted within the buffer space 1306 to be module 16 aligned. The origin 
1308 of the boundary for the original HTML image may not be modulo 16 
aligned. Since the origin and size of the displayed MPEG image at the client 
20 device must be modulo 1 6, the origin and size of the original HTML image are 
adjusted as shown in Figure 13 to modulo 16. The layout engine takes into 
account the larger buffer size, which is larger than the original HTML image, 
to avoid overlapping other page elements on the client display. This modulo 
16 buffer space, however, makes the image larger than original size. Thus, the 
25 buffer size is taken into account by the lay out engine so that the larger image 
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size does not overlap text or other imagery adjacent the image. The MPEG 
representation enables the present invention to move MPEG images around on 
a MPEG background for display on the client device. Thus, the layout engine 
may not strictly follow the HTML layout to avoid overlap in the client display 
5 space. » 

The present invention has been described in interactive television in a 
preferred embodiment, however, the present invention may also be embodied 
in a distributed computer system comprising a server and a client device. In 
another embodiment, the present invention is implemented as a set of 
1 0 instructions on a computer readable medium, comprising ROM, RAM, CD 
ROM, Flash or any other computer readable medium, now known or unknown 
that when executed cause a computer to implement the method of the present 
invention. 
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What is claimed is: 



1 1 . A computer readable medium containing instructions that when 

2 executed by a distributed computer system cause the distributed 

3 computer system to: 

4 receive content at a server from a service provider containing 

5 an original image; 

6 extract HTML original image information from the content; 

7 pass the original image size information to a Lay Out Engine; 

8 position the original image in the Lay Out Engine for display 

9 on a client device display to avoid overlapping content 

10 adjacent the image when displayed on the client device 

1 1 display; and 

12 send the positioned image to the client device. 

1 2. The medium of claim 1 further comprising instructions that cause the 

2 distributed computer system to: 

3 prior to positioning the image for display on the client device 
« 

4 display, 

5 convert the original image to MPEG format to form an 

6 MPEG image and adding a pixel buffer to an x and y 

7 boundary dimension of the MPEG image, so that an 

8 MPEG image boundary x and y dimension is modulo 

9 1 6 for layout on the client device display. 
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1 3 . The medium of claim 2 further comprising instructions that cause the 

2 distributed computer system to : 

3 calculate the layout of the modulo 16 MPEG image on the 

4 client device display using the modulo 16 image 

5 boundary dimension instead of using the original image 

6 dimensions. 

1 4. The medium of claim 3 further comprising instructions that cause the 

2 distributed computer system to : 

3 calculate the MPEG image boundaries wherein X_original and 

4 Y_original are the original image x and y dimensions, 

5 X_modified_boundaiy and Y_modified_boundary are 

6 the dimensions of modified image boundary, wherein 

7 16 <= X_modified_boundary - X_original < 32 and 

8 16 <= Y_modified_boundaiy - Y_priginal < 32. 

1 5. The medium of claim 1 further comprising instructions that cause the 

2 distributed computer system to: 

3 perform a shape ratio conversion on the modulo 1 6 MPEG 

4 image so that the modulo 16 MPEG image displayed on 

5 the client device display resembles the image as it 

6 would appear when displayed on a personal computer 

7 monitor. 
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1 6. The medium of claim 1 further comprising instructions that cause the 

2 distributed computer system to: 

3 convert a cascading style sheet (CSS) into an element node 

4 tree; and 

5 transmit the element node tree to the client device. 

1 7. The medium of claim 1 further comprising instructions that cause the 

2 distributed computer system to: 

3 provide a client compatible object model for the client device. 

1 8. The medium of claim 1 further comprising instructions that cause the 

2 distributed computer system to: 

3 extract HTML image information from the content in a MIME 

4 handler, wherein the MIME handler extracts the image 

5 from the content and converts the image to a MPEG 

6 encoded image, wherein the MIME handler adds a 

7 pixel buffer to the x and y dimension of the MPEG 

8 image so that the MPEG image size in the x and y 

9 dimensions is modulo 1 6, wherein the MIME handler 

10 passes the MPEG encoded image size information to a 

1 1 Lay Out Engine wherein the Lay Out Engine moves 

1 2 images around to avoid overlapping text adjacent an 

1 3 image on the client display. 
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1 9. The medium of claim 8 further comprising instructions that cause the 

2 distributed computer system to: 

3 perform a shape ratio conversion in the MIME handler on the 

4 image data so that the image displayed on the client 

5 device resembles the image as it would appear when 

6 displayed on a personal computer monitor. 

1 10. The medium of claim 8 further comprising instructions that cause the 

2 distributed computer system to: 

3 convert a cascading style sheet (CSS) into an element node 

4 tree; and 

5 transmit the element node tree to the client device. 

1 11. The medium of claim 1 further comprising instructions that cause the 

2 distributed computer system to: 

3 send service provider content to the server a head end. 

1 12. The medium of claim 1 further comprising instructions that cause the 

2 distributed computer system to: 

3 display the positioned MPEG image on the client device to a 

4 user. 

1 13. Hie medium of claim 1 2 further comprising instructions that cause the 

2 distributed computer system to: 
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complete an e-commerce transaction between the service 
provider and the user. 



1 14. A computer readable medium containing instructions that when 

2 executed by a distributed computer system cause the distributed 

3 computer system to: 

4 receive content at a server from a service provider containing 

5 an original image; 

6 extract HTML original image information from the content; 

7 pass the original image size information to a Lay Out Engine; 

8 convert the original image to MPEG format to form an MPEG 

9 image and adding a pixel buffer to an x and y boundary 

10 dimension of the MPEG image, so that an MPEG 

1 1 image boundary x and y dimension is modulo 1 6 for 

12 layout on the client device display; 

1 3 calculate the layout of the modulo 1 6 MPEG image on the 

14 client device display using the modulo 16 image 

1 5 boundary dimension instead of using the original image 

16 dimensions; 

1 7 position the original image in the Lay Out Engine for display 

18 on a client device display to avoid overlapping content 

1 9 adjacent the image when displayed on the client device 

20 display; and 

21 send the positioned image to the client device for display to a 

22 user. 
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1 15. The medium of claim 14 further comprising instructions that cause the 

2 distributed computer system to: 

3 calculate the MPEG image boundaries wherein X_originaI and 

4 Y_original are the original image x and y dimensions, 

5 Xjnodifiedjxmndary and Y_modified_boundary are 

6 the dimensions of modified image boundary, wherein 

7 16 <= X jnodified_boundary - X_original < 32 and 

8 16 <= Y_modified_boundary - Y_original < 32. 

1 16. The medium of claim 14 further comprising instructions that cause the 

2 distributed computer system to: 

3 perform a shape ratio conversion on the modulo 16 MPEG 

4 image so that the modulo 16 MPEG image displayed on 

5 the client device display resembles the image as it 

6 would appear when displayed on a personal computer 

7 monitor. 

1 17. The medium of claim 1 4 further comprising instructions that cause the 

2 distributed computer system to: 

3 convert a cascading style sheet (CSS) into an element node 

4 tree; and 

5 transmit the element node tree to the client device. 

1 18. The medium of claim 14 further comprising instructions that cause the 

2 distributed computer system to : 
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3 send service provider content from a head end to the server; 

4 display the modulo 16 MPEG image at a user at the client 

5 device; and 

6 complete an e-commerce transaction between the service 

7 provider and the client device user. 

1 19. A computer readable medium containing instructions that when 

2 executed by a distributed computer system cause the distributed 

3 computer system to: 

4 receive content at a server from a service provider containing 

5 an original image; 

6 extract HTML original image information from the content; 

7 pass the original image size information to a Lay Out Engine; 

8 convert the original image to MPEG format to form an MPEG 

9 image and adding a pixel buffer to an x and y boundary 

1 0 dimension of the MPEG image, so that an MPEG 

1 1 image boundary x and y dimension is modulo 16 for 

12 layout on the client device display; 

13 calculate the layout of the modulo 16 MPEG image on the 

14 client device display using the modulo 16 image 

1 5 boundary dimension instead of using the original image 

1 6 dimensions, wherein X_original and Y_original are the 

1 7 original image x and y dimensions, 

1 8 X_modified_boundary and Y_modified_boundary are 

1 9 the dimensions of modified image boundary, wherein 
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20 16 <= X_modified_boundary - X_original < 32 and 

21 1 6 <= Y_modified_boundary - Y_original < 32; 

22 position the original image in the Lay Out Engine for display 

23 on a client device display to avoid overlapping content 

24 adjacent the image when displayed on the client device 

25 display; 

26 send the positioned image to the client device for display to a 

27 user, 

28 convert a cascading style sheet (CSS) into an element node 

29 tree; and 

30 transmit the element node tree to the client device. 

1 20. The medium of claim 19 further comprising instructions that cause the 

2 distributed computer system to: 

3 send the service provider content from a head end to the server; 

4 and 

5 complete an e-commerce transaction between the service 

6 provider and the client device user. 

1 21. A method for formatting content for display in a distributed computer 

2 system comprising: 

3 receiving content at a server from a service provider containing 

4 an original image; 

5 extracting HTML original image information from the content; 

6 the HTML information comprising image size information; 
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7 passing the original image size information to a Lay Out 

8 Engine; 

9 positioning the original images in the Lay Out Engine for 

10 display on a client device display, based on the size 

1 1 information to avoid overlapping content text adjacent 

12 the image when displayed on the client device display; 

13 and 

14 sending the positioned images to the client device for display. 

1 22. The method of claim 2 1 further comprising: 

2 prior to positioning the image for display on the client device 

3 display, converting the content original image to MPEG 

4 format to form an MPEG image; and 

5 adding a pixel buffer to an the x and y boundary dimension of 

6 the MPEG se types of image, so that the MPEG image 

7 boundary in the x and y dimensions is module 1 6 for 

8 layout purpose on the client device display, while the 

9 actual image size remains as original. 

1 23. The method of claim 22, further comprising: 

2 calculating the layout of the modulo 1 6 MPEG image on the 

3 client device display using The Lay Out Engine uses 

4 the modulo 1 6 image boundary dimension in layout 

5 calculation instead of using the actual original image 

6 size dimensions. 
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1 24. The method of claim 23, further comprising: 

2 calculating the MPEG image boundaries wherein Let 

3 X_original and Y_original are the be image's original 

4 image x and y dimensions, X_modified_boundary and 

5 Y_modifiedJ)oundary are the dimensions of modified 

6 image boundary, wherein. The following are their 

7 relationships: 

8 16<=X_modifiedJ)Oundary - X_original < 32 and 

9 16 <= YjmodifiedjDOundary - Y_original < 32. 

1 23 . The transcoder of claim 2 where in the MIME handler extracts an 

2 image and converts the image to a MPEG encoded image. 

1 24. The MIME handler of claim 23 wherein the MIME handler adds a 

2 pixel buffer to the x and y dimension of the MPEG image so that the 

3 MPEG image size in the x and y dimensions is modulo 16. 

1 25. The MIME handler method of claim 24 further comprising: 

2 wherein the MIME handler performing s a shape ratio 

3 conversion on the modulo 1 6 MPEG image data so that 

4 the modulo 1 6 MPEG image displayed on the client 

5 device display resembles the image as it would appear 

6 when displayed on a personal computer monitor. 

1 26. The transcoder method of claim 21 further comprising: 
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2 




wherein the transcoder converting a cascading style sheet 


3 




(CSS) into an element node tree; and 


4 




transmitting the element node tree to the client device. 


1 


27. 


The transcoder method of claim 21 further comprising: 


2 




providing a client compatible object model for the client 


3 




device. 


1 


28. 


The transcoder method of claim 21 further comprising: 


2 




a MIME handler for extracting HTML image information from 


3 




the content in a MIME handler, wherein the MIME 


4 




handler extracts the image from the content and 


5 




converts the image to a MPEG encoded image, wherein 


6 




the MIME handler adds a pixel buffer to the x and y 


7 




dimension of the MPEG image so that the MPEG 


8 




image size in the x and y dimensions is modulo 16, 


9 




wherein the MIME handler passes the MPEG encoded 


10 




image size information to a Lay Out Engine wherein 


11 




the Lay Out Engine moves images around to avoid 


12 




overlapping text adjacent an image on the client 


13 




display. 


1 


29. 


The MIME handler of claim 28 wherein the MIME handler performs a 


2 




shape ratio conversion on the image data so that the image displayed 






47 



WO 02/063878 PCT/US02/02664 

3 on the client device resembles the image as it would appear when 

4 displayed on a personal computer monitor. 

1 • 30. The MIME handler of claim 28, further comprising a client compatible 

2 object model for the client device, wherein the transcoder converts a 

3 cascading style sheet (CSS) into an element node tree and transmits the 

4 element node tree to the client device. 

1 31. The method of claim 2 1 further comprising: 

2 sending service provider content to the server a head end. 

1 32. The method of claim 2 1 further comprising: 

2 displaying the positioned MPEG image on the client device to a 

3 user. 

1 33. The method of claim 32 further comprising: 

2 completing an ecommerce transaction between the service 

3 provider and the user. 

1 34. A method for formatting content for display in a distributed computer 

2 system comprising: 

3 receiving content at a server from a service provider containing 

4 an original image; 

5 extracting HTML original image information from the content; 
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6 passing the original image size information to a Lay Out 

7 Engine; 

8 converting the original image to MPEG format to form an 

9 MPEG image and adding a pixel buffer to an x and y 

1 0 boundary dimension of the MPEG image, so that an 

11 MPEG image boundary x and y dimension is modulo 

12 1 6 for layout on the client device display; 

1 3 calculating the layout of the modulo 1 6 MPEG image on the 

14 client device display using the modulo 16 image 

1 5 boundary dimension instead of using the original image 

16 dimensions; 

1 7 positioning the original image in the Lay Out Engine for 

1 8 display on a client device display to avoid overlapping 

1 9 content adjacent the image when displayed on the client 

20 device display; and 

2 1 sending the positioned image to the client device for display to 

22 a user. 

1 35. The method of claim 34, further comprising: 

2 calculating the MPEG image boundaries wherein X_original 

3 and Y_original are the original image x and y 

4 dimensions, X_modified_boundary and 

5 Y jnodified_boundary are the dimensions of modified 

6 image boundary, wherein 

7 16<=Xjtnodified_boundary - X_original < 32 and 
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8 16<= Y_modifiedJ)oundary - Y__original < 32. 

1 36. The method of claim 34 further comprising: 

2 performing a shape ratio conversion on the modulo 16 MPEG 

3 image so that the modulo 16 MPEG image displayed on 

4 the client device display resembles the image as it 

5 would appear when displayed on a personal computer 

6 monitor. 

1 37. The method of claim 34 further comprising: 

2 converting a cascading style sheet (CSS) into an element node 

3 tree; and 

4 transmitting the element node tree to the client device. 

1 38. The method of claim 34 further comprising: 

2 sending service provider content from a head end to the server; 

3 displaying the modulo 16 MPEG image at a user at the client 

4 device; and 

5 completing an ecommerce transaction between the service 

6 provider and the client device user. 

1 39. A method for formatting content for display in a distributed computer 

2 system comprising: 

3 receiving content at a server from a service provider containing 

4 an original image; 
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5 extracting HTML original image information from the content 

6 passing the original image size information to a Lay Out 

7 Engine; 

8 converting the original image to MPEG format to form an 

9 MPEG image and adding a pixel buffer to an x and y 

10 boundary dimension of the MPEG image, so that an 

11 MPEG image boundary x and y dimension is modulo 

12 1 6 for layout on the client device display; 

1 3 calculating the layout of the modulo 1 6 MPEG image on the 

14 . client device display using the modulo 16 image 

1 5 boundary dimension instead of using the original image 

1 6 dimensions, wherein X_original and Y_original are the 

1 7 original image x and y dimensions, 

18 X_modified_boundary and Y_modified_boundary are 

19 the dimensions of modified image boundary, wherein 

20 16 <= X_modified_boundary - X_original < 32 and 

21 16<= Y_modified_boundary - Y_original< 32; 

22 positioning the original image in the Lay Out Engine for 

23 display on a client device display to avoid overlapping 

24 content adjacent the image when displayed on the client 

25 device display; 

26 sending the positioned image to the client device for display to 

27 a user; 

28 converting a cascading style sheet (CSS) into an element node 

29 tree; and 
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30 transmitting the element node tree to the client device. 

1 40. The method of claim 3 9 further comprising: 

2 sending the service provider content from a head end to the 

3 server; and 

4 completing an ecommerce transaction between the service 

5 provider and the client device user. 

1 41. A method for formatting content for display in an interactive television 

2 system comprising: 

3 receiving content at a server from a service provider containing 

4 an original image; 

5 extracting HTML original image information from the content; 

6 passing the original image size information to a Lay Out 

7 Engine; 

8 positioning the original image in the Lay Out Engine for 

9 display on a client device display to avoid overlapping 

1 0 content adjacent the image when displayed on the client 

11 device display; and 

12 sending the positioned image to the client device. 

1 42. The method of claim 41 further comprising: 

2 prior to positioning the image for display on the client device 

3 display, converting the original image to MPEG format 

4 to form an MPEG image and adding a pixel buffer to an 
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5 x and y boundary dimension of the MPEG image, so 

6 that an MPEG image boundary x and y dimension is 

7 modulo 16 for layout on the client device display. 

1 43 . The method of claim 42, further comprising: 

2 calculating the layout of the modulo 1 6 MPEG image on the 

3 client device display using the modulo 1 6 image 

4 boundary dimension instead of using the original image 

5 dimensions. 

1 44. The method of claim 43, further comprising: 

2 calculating the MPEG image boundaries wherein X_original 

3 and Y_original are the original image x and y 

4 dimensions, X_modified_boundary and 

5 Y_modified_boundaiy are the dimensions of modified 

6 image boundary, wherein 

7 16 <= X_modifiedJx>undary - X_original < 32 and 

8 16 <= Y_modified_boundary - Y_original < 32. 

1 45 . The method of claim 4 1 further comprising: 

2 performing a shape ratio conversion on the modulo 1 6 MPEG 

3 image so that the modulo 1 6 MPEG image displayed on 

4 the client device display resembles the image as it 

5 would appear when displayed on a personal computer 

6 monitor. 
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1 46. The method of claim 41 further comprising: 

2 converting a cascading style sheet (CSS) into an element node 

3 tree; and 

4 transmitting the element node tree to the client device. 

1 47 . The method of claim 4 1 further comprising: 

2 providing a client compatible object model for the client 

3 device. 

1 48 . The method of claim 4 1 further comprising: 

2 extracting HTML image information from the content in a 

3 MME handler, wherein the MIME handler extracts the 

4 image from the content and converts the image to a 

5 . MPEG encoded image, wherein the MIME handler 

6 adds a pixel buffer to the x and y dimension of the 

7 MPEG image so that the MPEG image size in the x and 

8 y dimensions is modulo 16, wherein the MIME handler 

9 passes the MPEG encoded image size information to a 

1 0 Lay Out Engine wherein the Lay Out Engine moves 

1 1 images around to avoid overlapping text adjacent an 

12 image on the client display. 

1 49. The MIME handler of claim 48 wherein the MIME handler performs a 

2 shape ratio conversion on the image data so that the image displayed 
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3 on the client device resembles the image as it would appear when 

4 displayed on a personal computer monitor. 

1 50. The MIME handler of claim 48, further comprising a client compatible 

2 object model for the client device, wherein the transcoder converts a 

3 cascading style sheet (CSS) into an element node tree and transmits the 

4 element node tree to the client device. 

1 51. The method of claim 4 1 further comprising: 

2 sending service provider content to the server a head end. 

1 52. The method of claim 41 further comprising: 

2 displaying the positioned MPEG image on the client device to a 

3 user. 

1 53 . The method of claim 52 further comprising: 

2 completing an ecommerce transaction between the service 

3 provider and the user. 

1 54. A method for formatting content for display in an interactive television 

2 system comprising: 

3 receiving content at a server from a service provider containing 

4 an original image; 

5 extracting HTML original image information from the content; 
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6 passing the original image size information to a Lay Out 

7 Engine; 

8 converting the original image to MPEG format to form an 

9 MPEG image and adding a pixel buffer to an x and y 

10 boundary dimension of the MPEG image, so that an 

11 MPEG image boundary x and y dimension is modulo 

12 1 6 for layout on the client device display; 

13 calculating the layout of the modulo 16 MPEG image on the 

14 client device display using the modulo 1 6 image 

1 5 boundary dimension instead of using the original image 

16 dimensions; 

1 7 positioning the original image in the Lay Out Engine for 

1 8 display on a client device display to avoid overlapping 

19 content adjacent the image when displayed on the client 

20 device display; and 

21 sending the positioned image to the client device for display to 

22 a user. 

1 55. The method of claim 54, further comprising: 

2 calculating the MPEG image boundaries wherein X_priginal 

3 and Y_original are the original image x and y 

4 dimensions, X_modified ^boundary and 

5 YjmodifiedJ)oundary are the dimensions of modified 

6 image boundary, wherein 

7 16 <= X_modified_boundary - X_original < 32 and 
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8 1 6 <= Y_modified_boundary - Y_original < 32. 

1 56. The method of claim 54 further comprising: 

2 performing a shape ratio conversion on the modulo 1 6 MPEG 

3 image so that the modulo 16 MPEG image displayed on 

4 the client device display resembles the image as it 

5 would appear when displayed on a personal computer 

6 monitor. 

1 57. The method of claim 54 further comprising: 

2 converting a cascading style sheet (CSS) into an element node 

3 tree; and 

4 transmitting the element node tree to the client device. 

1 58. The method of claim 54 further comprising: 

2 sending service provider content from a head end to the server; 

3 displaying the modulo 1 6 MPEG image at a user at the client 

4 device; and 

5 completing an ecommerce transaction between the service 

6 provider and the client device user. 

1 59. A method for formatting content for display in an interactive television 

2 system comprising: 

3 receiving content at a server from a service provider containing 

4 an original image; 
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5 extracting HTML original image information from the content; 

6 passing the original image size information to a Lay Out 

7 Engine; 

8 converting the original image to MPEG format to form an 

9 MPEG image and adding a pixel buffer to an x and y 

10 boundary dimension of the MPEG image, so that an 

11 MPEG image boundary x and y dimension is modulo 

12 1 6 for layout on the client device display; 

1 3 calculating the layout of the modulo 1 6 MPEG image on the 

14 client device display using the modulo 16 image 

1 5 boundary dimension instead of using the original image 

1 6 dimensions, wherein Xj>riginal and Y_priginal are the 

1 7 original image x and y dimensions, 

1 8 Xjnodified_boundary and Yjnodified_boundary are 

19 the dimensions of modified image boundary, wherein 

20 16 <= X_modified_boundary - X_original < 32 and 

21 16 <= Y_modified_boundary - Y_original < 32; 

22 positioning the original image in the Lay Out Engine for 

23 display on a client device display to avoid overlapping 

24 content adjacent the image when displayed on the client 

25 device display; 

26 sending the positioned image to the client device for display to 

27 a user; 

28 converting a cascading style sheet (CSS) into an element node 

29 tree; and 
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30 transmitting the element node tree to the client device. 

1 60. The method of claim 59 further comprising: 

2 sending the service provider content from a head end to the 

3 server; and 

4 completing an ecommerce transaction between the service 

5 provider and the client device user. 

1 61 . An apparatus for formatting content for display in an interactive 

2 television system comprising: 

3 a server for receiving content at a server from a service 

4 provider containing an original image; 

5 an extraction component for extracting HTML original image 

6 information from the content and passing the original 

7 image size information to a Lay Out Engine; 

8 a positioning component for positioning the original image in 

9 the Lay Out Engine for display on a client device 

10 display to avoid overlapping content adjacent the image 

1 1 when displayed on the client device display; and 

12 a communication component for sending the positioned image 

13 to the client device. 

1 62. The apparatus of claim 6 1 further comprising: 

2 a conversion component for prior to positioning the image for 

3 display on the client device display, converting the 
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4 original image to MPEG format to form an MPEG 

5 image and adding a pixel buffer to an x and y boundary 

6 dimension of the MPEG image, so feat an MPEG 

7 image boundary x and y dimension is modulo 16 for 

8 layout on the client device display. 

1 63. The apparatus of claim 62, further comprising: 

2 a calculation component for calculating the layout of the 

3 modulo 16 MPEG image on the client device display 

4 using the modulo 16 image boundary dimension instead 

5 of using the original image dimensions. 

1 64. The apparatus of claim 63, wherein the calculation component further 

2 comprises a component for calculating the MPEG image boundaries 

3 wherein X_original and Y_original are the original image x and y 

4 dimensions, X_modified_boundary and Y_modified_boundary are the 

5 dimensions of modified image boundary, wherein 

6 16 <= X_modifiedJboundary - X_original < 32 and 

7 16 Y_modified_boundaiy - Y_original < 32. 

1 65. The apparatus of claim 61 further comprising: 

2 a shape ratio conversion component for performing a shape 

3 . ratio conversion on the modulo 1 6 MPEG image so that 

4 fee modulo 16 MPEG image displayed on the client 
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5 




device display resembles the image as it would appear 


6 




when displayed on a personal computer monitor. 


1 


66. 


The apparatus of claim 61 further comprising: 


2 




a cascading style sheet (CSS) conversion component for 


3. 




converting a CSS into an element node tree and 


4 




transmitting the element node tree to the client device. 


1 


67. 


The apparatus of claim 61 further comprising: 


2 




a client compatible object model for the client device. 


1 


68. 


The apparatus of claim 61 further comprising: 


2 




an extraction component for extracting a HTML image 


3 




information from the content in a MIME handler, 


4 




wherein the MIME handler extracts the image from the 


5 




content and converts the image to a MPEG encoded 


6 




image, wherein the MIME handler adds a pixel buffer 


7 




to the x and y dimension of the MPEG image so that 


8 




the MPEG image size in the x and y dimensions is 


9 




modulo 1 6, wherein the MIME handler passes the 


1U 




ivirJcAj encoaea image size miormauon to a Juay \jxx\ 


11 




Engine wherein the Lay Out Engine moves images 


12 




around to avoid overlapping text adjacent an image on 


13 




the client display. 
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1 69. The apparatus of claim 68, wherein the MIME handler performs a 

2 shape ratio conversion on the image data so that the image displayed 

3 on the client device resembles the image as it would appear when 

4 displayed on a personal computer monitor. 

1 70. The apparatus of claim 68 further comprising: 

2 a client compatible object model for the client device, wherein 

3 the transcoder converts a cascading style sheet (CSS) 

4 into an element node tree and transmits the element 

5 node tree to the client device. 

1 71. The apparatus of claim 6 1 further comprising: 

2 wherein the service provider content is sent to the server from a 

3 head end. 

1 72. The method of claim 6 1 further comprising: 

2 a client device display for displaying the positioned MPEG 

3 image on the client device to a user. 

1 73 . The apparatus of claim 72 wherein the element of an ecommerce 

2 transaction between the service provider and the user is displayed to 

3 the user. 

1 74. An apparatus for formatting content for display in an interactive 

2 television system comprising: 
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3 a server for receiving content at a server from a service 

4 provider containing an original image; 

5 an extraction component for extracting HTML original image 

6 information from the content and passing the original 

7 image size information to a Lay Out Engine; 

8 a conversion component for converting the original image to 

9 MPEG format to form an MPEG image and adding a 

10 pixel buffer to an x and y boundary dimension of the 

1 1 MPEG image, so that an MPEG image boundary x and 

12 y dimension is modulo 16 for layout on the client 

13 device display; 

14 a calculation component for calculating the layout of the 

1 5 modulo 1 6 MPEG image on the client device display 

1 6 using the modulo 1 6 image boundary dimension instead 

17 of using the original image dimensions; and 

18 a positioning component for positioning the original image in 

19 the Lay Out Engine for display on a client device 

20 display to avoid overlapping content adjacent the image 

21 when displayed on the client device display and 

22 sending the positioned image to the client device for display to 

23 a user. 

1 75 . The apparatus of claim 74, the calculation component further 

2 comprising: 
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3 a component for calculating the MPEG image boundaries 

4 wherein X_original and Y_original are the original 

5 image x and y dimensions, X_modified_boundary and 

6 Y_modified_boundary are the dimensions of modified 

7 image boundary, wherein 

8 16 <= X_modified_boundary - X_original < 32 and 

9 16 <= Y_modified_boundary - Y_priginal < 32. 

« 

1 76 . The method of claim 74 further comprising: 

2 a shape ratio conversion component for performing a shape 

3 ratio conversion on the modulo 16 MPEG image so that 

4 the modulo 16 MPEG image displayed on the client 

5 device display resembles the image as it would appear 
• 6 when displayed on a personal computer monitor. 

1 77. The apparatus of claim 74 further comprising: 

2 a cascading style sheet (CSS) conversion component for 

3 converting a CSS into an element node tree; and 

4 a transmission component for transmitting the element node 

5 tree to the client device. 

1 78 . The method of claim 74 further comprising: 

2 a head end for sending service provider content to the server; 

3 a client device display for displaying the modulo 16 MPEG 

4 image at a user at the client device and completing an 
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ecommerce transaction between the service provider 
and the client device user. 



1 79. 


A apparatus for formatting content for display in an interactive 


2 


television system comprising: 


3 


a server for receiving content at a server from a service 


4 


provider containing an original image; 


5 


an extraction component for extracting HTML original image 


6 


information from the content and passing the original 


7 


image size information to a Lay Out Engine, wherein a 


8 


conversion component in the Lay Out Engine converts 


9 


the original image to MPEG format to form an MPEG 


10 


image and adding a pixel buffer to an x and y boundary 


11 


dimension of the MPEG image, so that an MPEG 


12 


image boundary x and y dimension is modulo 1 6 for 


13 


layout on the client device display and calculate the 


14 


layout of the modulo 16 MPEG image on the client 


15 


device display using the modulo 16 image boundary 


lo 


dimension lnsieau 01 using me original image 


17 


dimensions, wherein X_original and Y_original are the 


18 


original image x and y dimensions, 


19 


Xjtnodified_boundary and Y_modified_boundary are 


20 


the dimensions of modified image boundary, wherein 


21 


16 <= X_modified_boundary - X_original < 32 and 16 


22 


<= Y_modifiedj30undary - Y_original < 32; 
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23 a positioning component for positioning the modulo 1 6 MPEG 

24 image in the Lay Out Engine for display on a client 

25 device display to avoid overlapping content adjacent 

26 the image when displayed on the client device display; 

27 sending the positioned modulo 16 MPEG image to the client 

28 device for display to a user, 

29 a cascading style sheet (CSS) conversion component for 

30 converting a CSS into an element node tree; and 

31 a transmission component for transmitting the element node 

32 tree to the client device. 

1 80. The apparatus of claim 79 further comprising: 

2 a head end for sending the service provider content to the 

3 server, wherein an ecommerce transaction between the 

4 service provider and the client device user is facilitated. 



66 



WO 02/063878 



PCT/US02/02664 



1/13 




Figure 1 



WO 02/063878 



PCT/US02/02664 



2/13 




Figure 2 



• 



WO 02/063878 



PCT7US02/02664 



3/13 



60 




Figure 3 



WO 02/063878 



PCT/US02/02664 



4/13 




Figure 4 



WO 02/063878 



PCT/US02/02664 



5/13 




WO 02/063878 PCT/US02/02664 



6/13 




WO 02/063878 



PCT/US02/02664 



7/13 



212 



1422 



246 



Customer Web Site 



amazoacom- + 




GateWay 



Compiled Object 
Cache 



JH20 Proxy 




'I 
8 
o 

LLI 

CL 



Not Compiled 
Object Cache 



1430 



50 



Service . 
Platform 




1424 



Figure 7 



WO 02/063878 



PCT/US02/02664 



8/13 




Figure 8 



WO 02/063878 



PCT/US02/02664 



9/13 




1670 Figure 9 



WO 02/063878 



PCT/US02/02664 



10/13 



424 



249 




H20 Proxy Host Software 



<D 

I 

Q. 



H20 Proxy 




HTML2RES Compiler 



GIF2PIX Compiler 



2MPEG Compiler 



1420 



1426 



1422 



Figure 10 



i 



WO 02/063878 



PCT/US02/02664 



11/13 



1820 




Figure 11 



WO 02/063878 



PCIYUS02/02664 



12/13 



1910 




1914 

Figure 12 



WO 02/063878 



13/13 



PCT/US02/02664 



1. Layout in Transcoden 



HTML page layout 



The origin of boundary have to conform to layout rule and 
not guaranteed to be modulo of 1 6, 

While the origin and size of displayed Image has to be modulo 16. 

it need an enlarged boundary 

to avoid overlapping other page elements. 



coordinate 
,0.0, 



Not align to modulo 16 
(19337) 



image 



8/^ 



130 



Latest development 



1304' 



Breaking News 



Source: CNN 




•4 - 

Reported by 




1309 



.1302 



"1306 



(589,397)^ 
(193+(16x25-*), 97 +(16x19-4) ) 
>image size Is not mod ulo of 16 



— (609.417)a 
(193+16x26, 97+16x20) 
boundary Is modul o of 16 



2. Presentation in Client Device: 



Expand mpeg size to modulo 16 



HTML page 



coordinate 
(0,0) 



Not align to modulo 16 Aligned Converted / image 
083,97) (208,112) Image 



1310- 



Latest development 




Bre a king News 



Source: CNN 




Reported by 



1312 




Figure 13 



(608,416)= 
(206+<16x25). 112+(16x19)) 
size Is modulo 16 



(609,417)s 
<193+16x28, 97+16x20) 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date 
15 August 2002 (15.08.2002) 




PCT 



(10) International Publication Number 

WO 02/063878 A3 



(51) International Patent Classification 7 : H04N 7/16, 

G06F 17/30, H04N 7/173 

(21) International Application Number: PCT/US02/02664 

(22) International Filing Date: 1 February 2002 (01.02.2002) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) 



Priority Data: 

60/265,986 
60/266,210 
60/267,876 
60/269,261 
60/279,543 



2 February 2001 (02.02.2001) US 

2 February 2001 (02.02.2001) US 

9 February 2001 (09.02.2001) US 

15 February 2001 (15.02.2001) US 

28 March 2001 (28.03.2001) US 



(71) Applicant: OPENTV, INC. [US/US]; 401 East Middle- 
field Road, Mountain View, CA 94043-4005 (US). 

(72) Inventor: SU, Dongmin, 2038 Finley Place, Santa Clara, 
CA 95050 (US). 



(74) Agent: ROEBUCK, G., Michael; Madan, Mossman & 
Sriram P.C., 2603 Augusta, Suite 700, Houston, TX 77057 
(US). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EC, EE, ES, FI, GB, GD, GE, GH, 
GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NO, NZ, OM, PH, PL, PT, RO, RU, SD, SE, SG, 
SI, SK, SL, TJ, TM, TN, TR, IT, TZ, UA, UG, UZ, VN, 
YU, ZA, ZM, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 
European patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, 
GB, GR, IE, IT, LU, MC, NL, PT, SE, TR), OAPI patent 
(BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, ML, MR, 
NE, SN, TD, TG). 

Published: 

— with international search report 

[Continued on next page] 



(54) Title: A METHOD AND APPARATUS FOR REFORMATTING OF CONTENT FIR DISPLAY ON INTERACTIVE TELE- 
VISION 



50 



< 
oo 

00 

m 

© 

is 




(57) Abstract: A method and apparatus for presenting a variety of content from a multitude of sources broadcast from a head end 
server to a client device. The broadcast content comprises HTML input from broadcast, cache, the internet or other sources is pro- 
cessed by a lay out engine for image positioning to avoid overlapping and shaped for appearance on a client device. The image is 
transferred to a client device in a client compatible code for presentation to a user on a client device. 



wo 02/063878 A3 I Bill ID1IIB H lillil lllil 111! I II in Hill Kill lllll 1110 Mill 1111 IllilO IHl 111! Illl 



(88) Date of publication of the international search report: 

5 December 2002 



For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



INTERNATIONAL SEARCH REPORT 



In* nal Application No 

PCT/US 02/02664 



A. CLASSIFICATION OF SUBJECT MATTER . , . 

IPC 7 H04N7/16 G06F17/30 H04N7/173 



According to International Patent Classification (IPC) or to both national classfflcalton end IPC 



a FIELDS SEARCHED 



Minimum documentation searched (classification system Mowed by dassification symbols) 

IPC 7 G06F H04N 



Documentation searched other than minimum documentation to the extent that such documents are included In the fields searched 



Electronic data base consulted during the International search (name of data base and, where practical, search terms used) 

EPO-IntGrnal 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category ° Citation of document, with Indication, where appropriate, of the relevant passages 



Relevant to claim No. 



EP 0 811 939 A (WEBTV NETWORKS INC) 
10 December 1997 (1997-12-10) 



abstract 

column 7, line 42 - line 53 

column 9, line 9 - line 25 

column 8, line 20 - line 34 



1,14,19, 

21,34, 

39,41, 

54,59, 

61,74,79 



-/- 



Further documents are listed in the continuation of box C. 



Patent family members are listed in annex. 



• Special categories of cited documents : 

•A* document defining the general state of the art which i3 not 

considered to be of particular relevance 
"E" earlier document but published on or alter the International 

filing date 

•L' document which may throw doubts on priority daim(s) or 
which Is cited to establish the publication date of another 
citation or other special reason (as specified) 

'O 1 document referring to an oral disclosure, use, exhibition or 
other means 

■P' document published prior to the international filing date but 
later than the priority date claimed 



T later document published after the International ftDng date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

*X* document of particular relevance; the ctafrned Invent ton 
cannot be considered novel or cannot be considered to 
involve an Inventive step when the document Is taken alone 

*Y' document of particular relevance; the claimed Invention 
cannot be considered to Involve an Inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skDJed 
In the art 

document member of the same patent famDy 



Dale of the actual completion of the international search 

6 September 2002 



Date of mailing of the Internationa) search report 



12/09/2002 



Name and mailing address of the ISA 

European Patent Office, P.8. 581 8 Patent laan 2 
NL - 22S0 HV R^swljk 
Tel. (+31-70) 340-2040. TX. 31 ©1 epo nl, 
Fax: (+31-70) 340-3016 



Authorized officer 



Dockhorn, H 



Foim PC17ISA/210 (second sheert) {July 1802} 



page- 1 or 2 



INTERNATIONAL SEARCH REPORT 


Intc iai Application No 

PCT/US 02/02664 


^Continuation) DOCUMENTS CONSIDERED TO BE RELEVANT 


Category 0 


Citation of document with Indication, where appropriate, of the relevant passages 


Relevant to claim No. 


A 


WO 00 39666 A (SPYGLASS) 
6 July 2000 (2000-07-06) 

abstract 

page 8, line 1 -page 10, line 5 
page 15, line 6 - line 17 
page 16, line 6 - line 14 
page 17, line 6 - line 11 




1,14,19, 

21,34, 

39,41, 

54,59, 

61,74,79 


A 


GB 2 346 238 A (IBM) 

2 August 2000 (2000-08-02) 

page 4, line 1 - line 18 
page 6, line 28 - line 31 
page 6, line 39 - line 42 
page 8, line 9 - line 10 




1,14,19, 

21,34, 

39,41, 

54,59, 

61,74,79 


P,A 


EP 1 170 675 A (GAVOA LTD) 
9 January 2002 (2002-01-09) 

paragraph '0009! 

paragraphs ' 0020!-' 0024 ! ; figures 2-5 
paragraph '0031! 
paragraphs '0038!-'0041! 




1,14,19, 

21,34, 

39,41, 

54,59, 

61,74,79 


E 


EP 1 202 188 A (MISH0U CO LTD) 
2 May 2002 (2002-05-02) 

paragraphs '00171/0018! 




1,13,19, 

21,34, 

39,41, 

54,59, 

61,74,79 



Form PCT/ISA/210 (ocrtbuaJton of eaoond shoot) (July 1992) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 

tut^rmstJon on patent family members 



lnt< al Application No 

PCT/US 02/02664 



Patent document 
cited In search report 


Publication 
date 


Patent family 
member(s) 


Publication 
dcits 


EP 0811939 A 


10-12-1997 


US 


5918013 


A 


29-06-1999 




AU 


3375197 


A 


05-01-1998 




EP 


0811939 


A2 


10-12-1997 




JP 


10228437 


A 


25-08-1998 




KR 


274738 


Bl 


15-12-2000 




US 


2001003823 


Al 


14-06-2001 




WO 


9746943 


Al 


11-12-1997 




US 


6023268 


A 


08-02-2000 




US 


6230319 


Bl 


08-05-2001 




US 


6141693 


A 


31-10-2000 




US 


6330606 


Bl 


11-12-2001 




US 


6311207 


Bl 


30-10-2001 




US 


6332157 


Bl 


18-12-2001 




US 


6259442 


Bl 


10-07-2001 




US 


6308222 


Bl 


23-10-2001 




US 


5940074 


A 


17-08-1999 




us 


6073168 


A 


06-06-2000 




us 


5935207 


A 


10-08-1999 




us 


5996022 


A 


30-11-1999 




us 


5974461 


A 


26-10-1999 




us 


2002013812 


Al 


31-01-2002 



W0 0039666 A 06-07-2000 DE 19962192 Al 06-07-2000 

FI 992746 A 28-06-2000 

GB 2347329 A 30-08-2000 

JP 2000194612 A 14-07-2000 

SE 9904687 A 29-06-2000 

WO 0039666 Al 06-07-2000 



GB 2346238 


A 


02-08-2000 


us 


2002059344 Al 


16-05-2002 








JP 


2000222275 A 


11-08-2000 


EP 1170675 


A 


09-01-2002 


EP 


1170675 Al 


09-01-2002 


EP 1202188 


A 


02-05-2002 


EP 


1202188 A2 


02-05-2002 








JP 


2002202935 A 


19-07-2002 








us 


2002062396 Al 


23-05-2002 



Fomn PCTflSA/210 (patent tamty annex) (July 1892) 



